Enclosure  I 


Observations  on  the  National 
Aeronautics  and  Space 
Administration's  Performance  Plan 
for  Fiscal  Year  2000 


The  National  Aeronautics  and  Space  Administration’s  (NASA)  fiscal  year  2000  annual 
performance  plan  should  be  useful  to  decisionmakers.  It  provides  a  limited  picture  of 
intended  performance  across  the  agency,  a  general  discussion  of  strategies  and  resources  the 
agency  will  use  to  achieve  its  goals,  and  limited  confidence  that  performance  information  will 
be  credible.  An  example  of  a  positive  change  regarding  the  plan’s  presentation  of  strategies 
and  goals  is  the  discussion  on  NASA’s  objective  of  extending  the  use  of  Earth  Science 
research  for  national,  state,  and  local  application.  The  plan  links  that  objective  with  the 
achievement  of  three  performance  goals,  namely  having  at  least  one  Regional  Earth  Science 
Application  become  self-sustaining;  developing  at  least  two  new  data  products  for  routine 
decision-making  by  user  organizations;  and  implementing  at  least  5  joint  applications 
research  projects/partnerships  with  state  and  local  governments  in  remote  sensing 
applications.  Figure  1  highlights  the  plan’s  major  strengths  and  key  weaknesses  as  NASA 
seeks  to  make  additional  improvements  to  its  plan. 

Figure  1:  Maior  Strengths  and  Kev  Weaknesses  of  Fiscal  Year  2000  Annual  Performance  Plan 


Major  Strengths 

•  Shows  how  budgetary  resources  are  related  to  performance 

•  Provides  expanded  detail  on  performance  evaluations  and  identifies  specific  data  sources 
Key  Weaknesses 

•  Does  not  provide  clear  rationale  for  how  information  technology  related  strategies  and 
programs  contribute  to  achievement  of  performance  goals 

•  Does  not  include  procedures  for  verifying  and  validating  performance  data _ 


NASA’s  fiscal  year  2000  plan  represents  a  moderate  improvement  over  the  fiscal  year  1999 
plan  in  that  it  indicates  some  degree  of  progress  in  addressing  the  weaknesses  identified  in 
our  assessment  of  the  fiscal  year  1999  plan.  In  reviewing  the  fiscal  year  1999  plan,  we 
observed  that  the  plan  could  have  provided  a  clearer  picture  of  intended  performance  across 
the  agency,  did  not  fully  portray  how  strategies  and  resources  would  help  achieve 
performance  goals,  and  partially  provided  confidence  that  performance  information  would  be 
credible.  Among  improvements  in  the  fiscal  year  2000  plan  is  the  inclusion  of  (1)  performance 
objectives  or  targets  that  fully  or  partially  address  5  of  10  management  challenges  identified 
by  NASA’s  Office  of  the  Inspector  General  or  us,  (2)  a  crosswalk  to  relate  budget  categories 
with  performance  objectives  and  targets,  and  (3)  expanded  detail  on  internal  and  external 
evaluations  of  performance. 
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Subject:  Observations  on  the  National  Aeronautics  and  Space  Administration’s  Fiscal  Year 
2000  Performance  Plan 


As  you  requested,  we  have  reviewed  and  evaluated  the  fiscal  year  2000  performance  plans  for 
the  24  Chief  Financial  Officers  (CFO)  Act  agencies  that  were  submitted  to  Congress  as 
required  by  the  Government  Performance  and  Results  Act  of  1993  (Results  Act).  Enclosure  I 
to  this  letter  provides  our  observations  on  the  fiscal  year  2000  performance  plan  for  the 
National  Aeronautics  and  Space  Administration  (NASA).  Enclosure  II  lists  management 
challenges  we  and  NASA’s  Inspector  General  identified  that  face  the  agency  and  the 
applicable  goals  and  measures  in  the  fiscal  year  2000  annual  performance  plan.  Enclosure  III 
is  NASA’s  comments. 

Our  objectives  were  to  (1)  assess  the  usefulness  of  the  agency’s  plan  for  decisionmaking  and 
(2)  identify  the  degree  of  improvement  the  agency’s  fiscal  year  2000  performance  plan 
represents  over  the  fiscal  year  1999  plan.  Our  observations  were  generally  based  on  the 
requirements  of  the  Results  Act,  guidance  to  agencies  from  the  Office  of  Management  and 
Budget  (OMB)  for  developing  the  plan  (OMB  Circular  A-l  1,  Part  2),  our  previous  reports  and 
knowledge  of  NASA’s  operations  and  programs,  and  our  observations  on  NASA’s  fiscal  year 
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1999  performance  plan.  Our  summary  report  on  the  CFO  Act  agencies’  fiscal  year  2000  plans 
contains  a  complete  discussion  of  our  objectives,  scope,  and  methodology.1 

As  agreed,  unless  you  announce  the  contents  of  this  letter  earlier,  we  plan  no  further 
distribution  until  30  days  from  the  date  of  the  letter.  The  major  contributors  to  this  report  are 
listed  in  enclosure  IV.  Please  call  me  on  (202)  512-4841  if  you  or  your  staff  have  any  questions. 


Allen  Li 

Associate  Director,  Defense 
Acquisitions  Issues 


Enclosures  -  4 


1  Managing  for  Results:  Opportunities  for  Continued  Improvements  in  Agencies’  Performance  Plans  (GAO/GGD/AIMD-99-215, 
July  20,  1999). 
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NASA’s  Performance  Plan  Provides  a  Limited  Picture  of 
Intended  Performance  Across  the  Agency 

The  plan  provides  a  limited  picture  of  intended  performance  across  the  agency.  The  plan 
includes  performance  goals  and  objectives  that  are  results  oriented,  including  the  provision 
of  performance  objectives  and  measures  that  address  important  dimensions  of  program 
performance  and  balance  competing  priorities.  Although  the  plan  includes  some  performance 
objectives  and  targets  that  address  mission-critical  management  problems,  the  plan  does  not 
address  all  nuyor  management  challenges  identified  by  NASA’s  Office  of  Inspector  General 
and  us.  It  attempts  to  address  issues  of  broad  national  concern  but  needs  more  specific 
information  on  how  achieving  the  agency’s  goals  will  contribute  to  addressing  these  issues. 

The  plan  includes  performance  goals  and  objectives  that  are  results  oriented.  Some  of  these 
goals  and  measures  address  important  dimensions  of  program  performance  and  balance 
competing  priorities.  For  example,  NASA’s  objective  of  providing  safe  and  affordable  access 
to  space  is  measured  by  safety  and  mission-related  performance  targets,  namely  achieving 
seven  or  fewer  anomalies  per  Space  Shuttle  mission  while  attaining  85  percent  on-time, 
successful  launches. 

While  the  plan  includes  some  performance  objectives  and  targets  that  address  mission- 
critical  management  problems,  it  does  not  address  all  10  of  the  major  management  challenges 
that  NASA’s  Inspector  General  and  we  identified.  The  plan  includes  performance  objectives 
or  targets  that  fully  or  partially  address  5  of  the  10  management  challenges.  Table  1  (in  the 
section  on  management  challenges)  shows  our  analysis  of  mgjor  management  challenges  and 
performance  objectives  in  the  plan.  As  an  example,  we  reported  that  contract  management  is 
a  continuing  area  of  high  risk  and  that  until  NASA’s  financial  management  system  is 
operational,  performance  assessments  relying  on  cost  data  may  be  incomplete.  NASA’s  plan 
addresses  this  issue  by  establishing  a  performance  target  to  implement  new  financial  systems 
and  business  procedures,  including  the  installation  of  its  Integrated  Financial  Management 
System. 

The  plan  attempts  to  address  crosscutting  issues  of  broad  national  concern,  but  at  a  very  high 
level.  For  example,  the  plan  states  that  the  outcomes  of  its  activities  contribute  significantly 
to  the  achievement  of  America’s  goals  in  key  areas.  The  key  area  of  economic  growth  and 
security  is  then  linked  with  NASA’s  aeronautics  and  space  research  and  development  of 
technology.  NASA’s  stated  desired  outcome  is  to  keep  America  capable  and  competitive.  The 
addition  of  more  detailed  information  on  how  NASA’s  efforts  toward  achieving  this  goal  will 
contribute  to  economic  growth,  such  as  referring  to  the  collaborative  work  on  engine 
development  by  NASA  and  private  industry,  would  enhance  the  understanding  of  linkages. 

The  fiscal  year  2000  performance  plan  indicates  some  degree  of  progress  in  addressing  the 
weaknesses  that  we  identified  in  our  assessment  of  the  fiscal  year  1999  performance  plan  as 
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it  relates  to  providing  a  clear  picture  of  intended  performance  across  the  agency.  Specifically, 
in  analyzing  the  fiscal  year  1999  plan,  we  said  that  the  plan  could  be  improved  by  (1) 
acknowledging  major  management  challenges  and  associated  corrective  actions  and,  (2) 
identifying  activities  that  had  been  undertaken  to  address  issues  of  broad  national  concern 
and  the  contribution  of  the  agency’s  goals  toward  addressing  such  crosscutting  activities. 
NASA’s  final  fiscal  year  1999  performance  plan  (which  was  issued  after  our  assessment  of  the 
version  provided  to  Congress)  and  NASA’s  fiscal  year  2000  performance  plan  address  some  of 
the  weaknesses  we  cited  previously.1  Specifically,  the  final  fiscal  year  1999  performance  plan 
added  performance  targets  to  address  two  of  the  three  major  management  challenges  that  we 
identified,  and  one  of  those  two  appears  in  the  fiscal  year  2000  performance  plan.  NASA 
officials  indicated  that  the  remaining  management  challenge  did  not  appear  in  the  fiscal  year 
2000  plan  because  remedial  action  on  it  would  be  completed  in  fiscal  year  1999.  A  third 
management  challenge  that  we  identified — cooperation  between  the  Department  of  Defense 
and  NASA  on  using  aerospace  testing  facilities — does  not  appear  in  either  plan.  Although 
NASA  officials  said  that  they  did  not  specifically  attempt  to  address  the  major  management 
challenges  identified  by  NASA’s  Office  of  Inspector  General,  the  fiscal  year  2000  plan  does 
include  performance  objectives  or  targets  that  relate  to  some  of  these  management 
challenges.  For  example,  the  Office  of  Inspector  General  reported  that  ensuring  the 
availability  of  launch  vehicles  presented  challenges.  These  challenges  included  (1)  ensuring 
the  availability  of  small  expendable  launch  vehicles  so  that  milestones  can  be  met  and  NASA 
missions  are  cost-effective  and  (2)  evaluating  whether  NASA’s  providing  the  majority  of 
development  funds  and  assigning  technology  rights  to  its  industry  partners  in  the 
development  of  the  new  reusable  launch  vehicles  is  in  the  best  interest  of  the  government. 
The  plan  includes  an  objective  related  to  this  challenge  that  NASA’s  plan  characterizes  as 
revolutionizing  space  launch  capabilities.  Specifically,  it  provides  a  performance  target  to 
begin  and  complete  flight  testing  of  the  X-33  in  fiscal  year  2000  to  demonstrate  technologies 
required  for  future  reusable  launch  vehicles.  NASA  officials  also  noted  that  while  there  was 
no  conscious  effort  not  to  acknowledge  the  management  challenges,  NASA  was  still 
struggling  with  how  to  package  that  issue  in  its  performance  plan. 

NASA’s  Performance  Plan  Provides  a  General  Discussion  of  the 
Strategies  and  Resources  It  Will  Use  to  Achieve  Its  Goals 

The  plan  provides  a  general  discussion  of  strategies  and  resources  the  agency  will  use  to 
achieve  performance  goals.  The  plan  provides  a  crosswalk  that  relates  enterprises, 
performance  objectives  and  targets  to  the  program  activities  in  the  President’s  budget  request 


‘NASA  issued  its  final  fiscal  year  1999  performance  plan  in  February  1999.  NASA  officials  said  that  the  plan  was  updated  to 
reflect  fiscal  year  1999  appropriations  and  programmatic  changes  and  changes  made  to  respond  to  concerns  expressed  by 
outside  reviewers. 
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and  to  the  budget  categories  used  in  the  congressional  budget  justification;  more  completely 
discusses  resources,  with  the  addition  of  civil  service  staffing  levels  for  each  enterprise;  and 
provides  a  limited  discussion  of  external  factors  that  could  influence  the  successful 
achievement  of  performance  results.  However,  the  plan  still  does  not  adequately  describe 
how  the  agency’s  strategies  and  resources  will  help  it  achieve  performance  goals,  including  a 
clear  rationale  for  how  information  technology-related  strategies  and  programs  will 
contribute  to  the  achievement  of  performance  goals. 

NASA’s  plan  explicitly  states  that  the  plan  is  intended  to  be  used  in  coryunction  with  the 
congressional  budget  justification,  and  the  plan  does  relates  performance  objectives  and 
targets  to  budget  categories.  The  plan  adds  a  crosswalk  that  links  strategic  objectives  and 
performance  targets  to  the  budget  categories  used  in  NASA’s  congressional  budget 
justification.  This  crosswalk  indicates  that  NASA’s  performance  targets  cover  the  program 
activities  in  its  budget  justification.  For  example,  in  discussing  NASA’s  objective  of  extending 
the  use  of  Earth  Science  research  for  national,  state,  and  local  applications,  the  plan  relates 
this  objective  and  associated  resources  (identified  in  the  budget  justification)  to  its  strategic 
goal  of  enabling  the  productive  use  of  Earth  Science  and  technology  in  the  public  and  private 
sectors.  The  plan  links  the  objective  with  the  achievement  of  three  performance  goals, 
namely  having  at  least  one  Regional  Earth  Science  Application  become  self-sustaining; 
developing  at  least  two  new  data  products  for  routine  decision-making  by  user  organizations; 
and  implementing  at  least  5  joint  applications  research  projects/partnerships  with  state  and 
local  governments  in  remote  sensing  applications. 

Although  this  linkage  is  a  step  in  the  right  direction,  the  absence  of  funding  information  for 
the  budget  categories  makes  it  difficult  to  easily  relate  requested  funding  needs  to  the 
performance  objectives  and  targets.  Including  these  funding  needs  in  the  performance  plan 
would  permit  decisionmakers  to  more  readily  compare  planned  levels  of  accomplishment 
with  budget  resources.  Also,  the  plan  would  be  more  useful  if  NASA  had  provided  the 
meaning  and  significance  of  the  primary  and  secondary  contribution  notations  in  the 
crosswalk  and  an  explanation  of  acronyms  used  in  the  crosswalk  to  identify  other  objectives 
addressed  by  the  performance  targets.  NASA  recognizes  that  its  performance  plan  and  budget 
justification  need  to  be  integrated  to  better  relate  the  performance  goals,  objectives,  and 
measures.  According  to  agency  officials,  NASA  was  studying  the  benefits  of  merging  these 
documents. 

The  plan’s  use  of  performance  target  descriptions  and  numbers,  the  crosswalk,  and  the 
narrative  were  found  to  be  consistent  in  all  of  the  enterprise  sections  except  in  the  Space 
Science  Enterprise  section.  However,  to  minimize  repetition,  the  plan  could  be  streamlined 
by  referencing  key  targets  in  the  narrative  and  leaving  detailed  information  in  the  crosswalk. 

The  plan  provides  a  more  complete  discussion  of  resources  with  the  addition  of  civil  service 
staffing  levels  for  each  enterprise.  However,  there  is  no  further  description  of  NASA’s  human 
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capital  needs  beyond  this  discussion.  For  example,  skills  and  training  needed  to  meet 
performance  goals  are  not  addressed.  In  addition,  the  plan  still  does  not  adequately  describe 
how  the  agency’s  strategies  and  resources  will  help  it  achieve  performance  goals.  Generally, 
implementation  strategies  have  not  changed  from  the  fiscal  year  1999  plan  and  still  lack 
specific  information  on  how  NASA  will  accomplish  its  enterprise  goals.  In  several  instances, 
these  descriptions  outline  philosophies  rather  than  describe  enabling  actions.  For  instance, 
the  plan  still  does  not  clearly  explain  application  of  the  “faster,  better,  cheaper”  approach  to 
spacecraft  development  in  the  Space  Science  and  Earth  Science  implementation  strategy 
sections.  At  a  minimum,  it  would  have  been  helpful  if  the  plan  had  been  updated  to  include  an 
explicit  discussion  of  what  is  involved  in  applying  the  “faster,  better,  cheaper”  approach  and 
how  the  establishment  of  “prudent  risk”  is  part  of  it. 

NASA  officials  responded  that  the  performance  plan  is  intended  to  be  used  in  conjunction 
with  the  NASA  fiscal  year  2000  congressional  budget  justification  and  noted  that  each  of  the 
entries  in  the  budget  justification  identifies  resources,  goals,  and  the  strategies  being  pursued 
in  the  implementation  of  those  goals.  While  the  scope  of  our  analysis  did  not  include  a  full 
analysis  of  NASA’s  fiscal  year  2000  budget  justification,  we  found  the  information  provided  in 
that  document  not  to  be  useful  in  further  explaining  implementation  strategies  referenced  in 
the  performance  plan.  As  indicated  earlier,  NASA  officials  said  that  the  agency  is  studying  the 
benefits  of  merging  the  performance  plan  and  the  congressional  budget  justification  in  part  to 
address  the  strategies  and  resources  issue. 

Relative  to  information  technology  (IT),  the  plan  generally  does  not  provide  a  clear  rationale 
for  how  IT-related  strategies  and  programs  will  contribute  to  the  achievement  of  its 
performance  goals,  nor  does  it  show  the  allocation  of  IT-related  dollars  and  personnel  to 
performance  goals.  The  plan  states  that  costs  for  mission-unique  IT  support  are  incorporated 
in  program  and  project  budgets.  Therefore,  they  are  not  identified  as  separate  items. 
Furthermore,  we  believe  that  the  plan  is  unrealistic  in  its  discussion  of  how  the  Earth 
Observing  System  Data  and  Information  System  (EOSDIS),  an  IT-related  resource,  will 
contribute  to  the  achievement  of  NASA’s  performance  goal  of  increasing  the  number  of 
products  delivered  from  its  archive  and  make  the  data  available  to  users  more  rapidly  in  fiscal 
year  2000.  NASA  has  already  testified  before  Congress  that  repairing  the  problems  with 
EOSDIS  that  resulted  in  the  delay  of  the  launch  of  the  AM-1  satellite  and  staying  within 
budget  will  require  scaling  back  some  of  the  system’s  capabilities. 

The  plan  provides  additional  context  regarding  anticipated  performance  by  including  a  brief 
discussion  of  external  factors  that  could  influence  the  successful  achievement  of 
performance  results.  For  example,  the  plan  states  that  the  successful  execution  of  NASA’s 
strategic  goals  and  objectives  is  contingent  on  receipt  of  the  requested  appropriations,  and 
the  provision  of  funds,  materials,  or  services  that  have  been  committed  to  the  cooperative 
agreements  or  partnerships  that  are  referenced  in  the  plan.  However,  the  value  of  the  plan 
would  be  augmented  if  (1)  external  factors  were  explicitly  discussed  relative  to  objectives 
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whose  attainment  is  strongly  tied  to  these  external  influences  and  (2)  NASA  explained  how  it 
would  mitigate  any  negative  effects. 

The  fiscal  year  2000  performance  plan  indicates  some  degree  of  progress  in  addressing  the 
weaknesses  that  we  identified  in  our  assessment  of  the  fiscal  year  1999  performance  plan  as 
it  relates  to  providing  a  specific  discussion  of  strategies  and  resources  the  agency  will  use  to 
achieve  performance  goals.  In  reviewing  the  fiscal  year  1999  plan,  we  observed  that  the  plan 
(1)  did  not  clearly  associate  performance  goals  and  measures  with  program  activities, 
including  not  providing  a  crosswalk  to  facilitate  an  understanding  of  the  relationship  between 
the  specific  program  activities  in  the  budget  and  the  goals  and  measures  in  the  performance 
plan;  (2)  partially  discussed  the  resources  the  agency  would  use  to  achieve  the  performance 
goals;  (3)  did  not  fully  describe  how  the  agency  would  achieve  its  performance  goals  or 
reference  other  documents  to  show  how  the  strategies  would  contribute  to  achieving  the 
performance  goals;  (4)  did  not  address  any  of  the  important  IT  management  issues  or  include 
a  discussion  of  the  agency’s  strategy  for  achieving  improvements  in  IT  capability  and 
services;  and  (5)  could  have  provided  additional  context  through  a  discussion  of  external 
factors  and  the  agency’s  plan  for  mitigating  or  using  the  identified  factors  to  achieve 
performance  goals.  The  fiscal  year  2000  plan  shows  some  progress  in  addressing  weaknesses 
that  we  identified.  For  example,  the  plan  provides  a  crosswalk  that  relates  enterprises, 
performance  objectives  and  targets  to  the  program  activities  in  the  President’s  budget  request 
and  to  the  budget  categories  in  the  congressional  budget  justification.  In  addition,  the  plan 
provides  a  more  complete  discussion  of  resources  with  the  addition  of  civil  service  staffing 
levels  for  each  enterprise. 

The  Agency’s  Performance  Plan  Provides  Limited  Confidence 
That  Agency  Performance  Information  Will  Be  Credible 

The  plan  provides  limited  confidence  that  the  agency’s  performance  information  will  be 
credible.  The  plan  identifies  internal  and  external  organizations  that  will  evaluate 
performance,  provides  expanded  detail  on  such  evaluations,  and  identifies  specific  internal 
and  external  sources  for  data.  However,  it  does  not  include  an  explicit  discussion  of 
procedures  that  will  be  used  to  verify  and  validate  performance  data.  Furthermore,  the  plan 
does  not  address  possible  limitations  in  internal  and  external  sources  of  data. 

The  plan  states  that  through  internal  and  external  processes  NASA  will  evaluate  performance 
and  that  these  reviews  will  provide  an  opportunity  to  verify  and  validate  performance  data 
provided  by  implementing  organizations.  The  plan  provides  expanded  detail  on  such 
evaluations.  In  addition  to  naming  the  specific  internal  and  external  entities  that  will  evaluate 
performance,  such  as  the  Board  of  Directors  of  the  Office  of  Life  and  Microgravity  Sciences 
and  Applications,  the  plan  identifies  specific  internal  and  external  sources  for  the  data.  By 
identifying  relevant  data  sources,  decisionmakers  can  be  informed  of  the  agency’s  actions  to 
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measure  performance.  For  example,  the  plan  states  that  NASA’s  Earth  Science  Enterprise 
will  use  a  database  to  track  the  monthly  performance  for  each  specific  fiscal  year  2000 
performance  target;  this  data  will  help  the  Enterprise  focus  on  targets  that  need 
improvement.  Enterprise  officials  told  us  that  target  achievability  is  rated  red  if  the  target  is 
at  risk  of  not  achieving  performance  results;  yellow  if  there  are  concerns  and  issues  that  need 
to  be  resolved;  or  green  if  the  target  is  on  track.  Enterprise  officials  told  us  that  this 
management  tool  is  already  being  used. 

However,  the  plan  does  not  include  an  explicit  discussion  of  procedures  that  will  be  used  to 
verify  and  validate  performance  data,  nor  does  it  address  limitations  to  the  data  from  internal 
or  external  sources.  NASA  officials  said  that  established  processes  have  no  known  limitations 
and  they  consider  the  issue  to  be  adequately  addressed  in  that  the  plan  explicitly  states  that 
internal  management  processes  provide  appropriate  forums  for  reporting  and  reviewing  of 
project  and  program  performance  data;  the  recent  streamlining  of  agency  processes  provide 
confidence  that  new  data  collection  and  oversight  processes  need  not  be  created  for 
compliance  with  the  Results  Act  and  NASA’s  mission-oriented  organizational  structure  and 
established  management  processes  are  well-suited  for  this  type  of  performance  evaluation; 
and  a  significant  in-place  external  review  process  involves  a  peer  review  process  as  a  key 
element  to  ensure  that  science  research  proposals  are  selected  strictly  on  the  merits  of  the 
planned  research. 

While  NASA  is  confident  that  established  processes  have  no  known  data  limitations, 
identification  of  possible  limitations  could  permit  it  to  identify  actions  that  might  compensate 
for  events  that  could  negatively  affect  achievement  of  performance  goals.  For  example, 
evaluation  of  NASA’s  performance  objective  of  improving  effectiveness  and  efficiency  of 
agency  acquisitions  is  dependent  on  data  from  the  agency’s  Financial  Accounting  and 
Contractual  Status  system.  Based  on  the  problems  NASA  has  encountered  in  producing 
complete  cost  data,  uncertain  reliability  or  unavailability  of  data  from  this  system  needs  to  be 
considered.  We  previously  reported  that  until  the  new  Integrated  Financial  Management 
System  is  operational,  performance  assessments  relying  on  cost  data  might  be  incomplete.2 
To  NASA’s  credit,  the  agency  has  recognized  the  importance  of  this  data  The  fiscal  year  2000 
plan  includes  a  performance  target  to  implement  new  financial  systems  and  business 
procedures  (including  a  new  financial  classification  structure  and  full  cost  budgeting  and 
accounting  procedures)  through  the  introduction  and  installation  of  the  Integrated  Financial 
Management  System.  The  plan  states  that  following  completion  of  system  testing,  the  agency 
will  begin  to  install  the  system  at  NASA  headquarters  and  field  centers.  Consideration  of  data 
limitations  needs  to  extend  to  external  data  sources  as  well.  NASA’s  performance  plan  does 


2  Major  Management  Challenges  and  Program  Risks:  National  Aeronautics  and  Space  Administration  (GAO/O CG-99- 18,  January 
1999). 
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not  specifically  address  data  limitations  of  externally  provided  data,  such  as  quality,  validity, 
and  timeliness.  For  example,  NASA  will  rely  on  Federal  Aviation  Administration  data  to 
ascertain  whether  its  goal  of  increasing  operations  throughput  was  achieved. 

The  fiscal  year  2000  plan  shows  some  degree  of  progress  in  addressing  the  weaknesses 
identified  in  our  assessment  of  the  fiscal  year  1999  plan  as  it  relates  to  providing  full 
confidence  that  the  agency’s  performance  information  will  be  credible.  In  reviewing  the  fiscal 
year  1999  plan,  we  observed  that  the  plan  did  not  (1)  include  a  discussion  of  the  procedures 
that  would  be  used  to  verify  and  validate  performance  data  and  (2)  address  limitations  to  the 
data  from  internal  or  external  sources.  Among  improvements  in  the  fiscal  year  2000  plan  are 
(1)  expanded  detail  on  internal  and  external  evaluations,  (2)  identification  of  specific  internal 
and  external  sources  for  data,  and  (3)  identification  of  planned  action  to  complete  testing  and 
begin  installation  of  the  Integrated  Financial  Management  System  at  NASA  headquarters  and 
field  centers  in  fiscal  year  2000. 

Other  Observations 

NASA  is  making  inroads  in  ensuring  that  its  strategic  management  system — which  includes 
its  strategic  and  annual  performance  plans  and  budget  justification— is  reflective  of  current 
conditions  and  agency  priorities.  NASA  revised  its  1998  strategic  plan  with  interim 
adjustments  released  as  part  of  the  fiscal  year  2000  performance  plan  and  plans  to  release  a 
fully  updated  strategic  plan  before  September  2000.  NASA  will  also  issue  a  revised  fiscal  year 
2000  performance  plan,  if  required.  Although  we  understand  NASA’s  rationale  for  wanting  to 
reflect  programs  for  which  appropriations  were  made  and  not  requested,  NASA’s  updated 
plans  applicable  to  the  current  fiscal  year  will  be  most  useful  to  decisionmakers  if  they  reflect 
progress  NASA  already  has  made  in  meeting  goals  in  its  prior  fiscal  year  plan.  For  example, 
the  final  version  of  the  fiscal  year  1999  plan  indicates,  within  the  “Manage  Strategically”  goal, 
a  performance  target  to  “enhance  contract  management  through  improved  systems  and 
information  for  monitoring  and  through  an  emphasis  on  training  of  procurement  personnel, 
and  revise  metrics  to  assess  the  overall  health  of  the  procurement  function.”  Since  the  target 
is  to  be  achieved  in  fiscal  year  1999,  it  is  not  included  in  the  fiscal  year  2000  plan.  Using  only 
the  fiscal  year  1999  plan  as  a  basis  for  comparison,  decisionmakers  are  unaware  of  NASA’s 
positive  action.  To  address  this  issue,  future  performance  plans  could  be  enhanced  with  the 
inclusion  of  an  appendix  summarizing  adjustments  made  to  the  prior  fiscal  year  plan  but  no 
longer  applicable  to  the  current  fiscal  year. 

Agency  Comments 

In  commenting  on  a  draft  of  our  analysis,  NASA’s  Associate  Deputy  Administrator  stated  that 
it  would  endeavor  to  use  our  observations  in  improving  the  management  of  the  agency.  NASA 
raised  concern  about  three  issues  that  we  identified  in  our  analysis.  First,  the  agency  stated 


Page  10 


GAO/NSIAD-99-186R  NASA’s  Fiscal  Year  2000  Performance  Plan 


Enclosure  I 

Observations  on  the  National  Aeronautics  and  Space  Administration’s  Performance  Plan  for  Fiscal  Year  2000 


that  the  report  containing  the  OIG’s  management  challenges  was  issued  subsequent  to 
NASA’s  formulation,  selection,  and  submittal  of  its  performance  targets  to  the  Office  of 
Management  and  Budget  (OMB).  NASA  contends  that  the  OIG’s  management  challenges  were 
identified  too  late  to  enable  inclusion  in  the  performance  plan.  Our  point  concerning  this 
matter  is  that  NASA  did  not  recognize  and  address  longstanding  major  management  issues 
that  the  agency  knew  about  or  should  have  known  about.  The  OIG  had  identified  similar 
challenges  in  a  report  to  Senator  McCain  in  1997.  These  issues  should  have  been  addressed  in 
the  plan  upon  submittal  to  OMB. 

Second,  in  response  to  our  criticism  that  the  plan  did  not  include  an  explicit  discussion  of 
external  factors  that  could  impact  attainment  of  the  plan’s  performance  objectives,  NASA 
stated  that  OMB  Circular  A-ll  allows  agencies  to  choose  to  include  external  factors  in  their 
plans  when  these  may  bear  directly  on  goal  achievement  for  the  fiscal  year  covered.  NASA  is 
correct  in  its  interpretation  that  the  OMB  regulations  do  not  require  inclusion  of  external 
factors  in  annual  performance  plans,  and  we  have  been  careful  not  to  state  that  they  are 
required  in  these  plans.  However,  the  focus  of  our  assessment  is  on  how  the  plan  can  be  more 
useful  to  Congress  and  other  decisionmakers.  NASA  provided  a  general  discussion  of 
external  factors  in  its  fiscal  year  2000  plan  that  were  not  specific  to  any  objective.  We 
continue  to  believe  that  the  plan  would  be  augmented  if  (a)  external  factors  were  explicitly 
discussed  relative  to  objectives  whose  attainment  is  strongly  tied  to  these  external  influences 
and  (b)  NASA  explained  how  it  would  mitigate  any  negative  effects. 

Third,  NASA  disagreed  with  our  characterization  of  its  plan  as  providing  a  “limited  picture  of 
intended  performance  across  the  agency.”  NASA  stated  that  its  plan  included  detailed 
performance  targets  that  cover  all  of  its  program  and  finance  accounts  and  most  of  its  budget 
line  items  and  programmatic  elements  of  its  strategic  plan.  Our  assessment  of  the  plan,  as  it 
related  to  providing  an  intended  picture  of  performance  across  the  agency,  focused  on 
whether  the  sum  total  of  goals,  objectives,  and  measures  provided  a  clear  understanding  and 
complete  picture  of  performance  across  the  agency,  not  on  the  number  of  performance 
targets  the  plan  provided.  The  incomplete  discussion  of  longstanding  management  issues  is 
illustrative  of  the  plan’s  need  to  address  longstanding  priority  issues,  a  key  indicator  of  NASA 
having  provided  a  complete  picture  of  performance  across  the  agency. 
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Table  II.  1  shows  NASA’s  major  management  challenges  that  we  and  NASA’s  Office  of  the 
Inspector  General  (OIG)  have  identified  and  NASA’s  annual  performance  plan  for  fiscal  year 
2000  addresses. 

Table  11.1 :  Management  Challenges  in  NASA’s  Fiscal  Year  2000  Performance  Plan 

GAO  identified  management  challenge 

Applicable  references  in  the  fiscal  year  2000 
annual  performance  plan 

Contract  manaqement 

GAO  has  reported  that  NASA’s  contract  management 
is  a  continuing  area  of  high  risk.  Implementation  of 
the  financial  management  system  and  its  integration 
with  full  cost  accounting  has  been  delayed.  Until  the 
financial  management  system  is  operational, 
performance  assessments  relying  on  cost  data  may 
be  incomplete. 

NASA’s  performance  plan  includes  a  performance 
target  to  implement  new  financial  systems  and 
business  procedures  (including  a  new  financial 
classification  structure  and  full  cost  budgeting  and 
accounting  procedures)  through  the  introduction  and 
installation  of  the  Integrated  Financial  Management 
System  (IFMS).  The  plan  notes  that  following 
completion  of  system  testing,  the  system  will  begin  to 
be  installed  at  NASA  headquarters  and  field  centers. 

NASA’s  OIG  reported  contract  management  as  a 
GAO-identified  management  challenge. 

The  final  FY  1999  performance  plan  also  added 
targets  to  describe  additional  procurement  actions 
being  undertaken  in  FY  1999.  It  stated  that  contract 
management  would  be  enhanced  through  improved 
systems  and  information  for  monitoring  and  through  an 
emphasis  on  the  training  of  procurement  personnel. 
Also,  metrics  would  be  revised  to  assess  the  overall 
health  of  the  procurement  system.  It  also  stated  that  a 
strategy  for  evaluating  the  efficacy  of  procurement 
operations  would  be  implemented  in  FY  1999.  (NASA 
officials  noted  that  these  specific  remedial  actions  will 
be  completed  in  FY  1999  and  thus  do  not  appear  in 
the  FY  2000  plan.) 

International  Space  Station  Program 

We  reported  that  the  International  Space  Station 
Program  continues  to  face  cost  and  schedule 
challenges  and  the  effects  of  funding  shortfalls  in 
Russia;  NASA  continues  to  identify  cost  growth  and 
limited  reserves  as  major  program  concerns  and  is 
now  giving  added  attention  to  problems  with  support 
from  others  than  the  prime  contractor.  In  January 

1999,  we  reported  that  the  prime  contractor’s  latest 
overrun  estimate  at  completion  has  increased  from 
$680  million  to  $780  million. 

None.  However,  the  management  of  risks  in  building 
the  Internationa!  Space  Station  is  identified  in  the 
President’s  FY  2000  budget  submission  as  one  of  the 
top  24  government  management  challenges. 

NASA’s  OIG  also  reported  that  the  International 

Space  Station  Program  continues  to  experience  cost 
overruns  and  scheduling  delays. 

CooDeration  between  NASA  and  Department  of 
Defense  on  aerospace  test  facilities 

None. 
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GAO  identified  management  challenge  Applicable  references  in  the  fiscal  year  2000 

_ annual  performance  plan _ 


We  reported  that  the  promise  of  closer  cooperation 
between  NASA  and  DOD  and  the  development  of  a 
national  perspective  on  aerospace  test  facilities 
remain  largely  unfulfilled.  NASA  and  DOD  (1)  have 
not  convened  most  joint  test  facility  working  groups 
on  a  regular  basis,  (2)  have  competed  with  each  other 
to  test  engines  for  new  rockets,  and  (3)  have  not 
prepared  a  congressionally  required  joint  plan  on 
rocket  propulsion  test  facilities. 

Inspector  General’s  areas  of  concern 

Applicable  references  in  the  fiscal  year  2000 
annual  performance  plan 

Safety  and  mission  assurance 

NASA’s  OIG  has  reported  that  safety  and  mission 
assurance  has  become  a  serious  challenge  to  NASA. 
The  challenges  are  ensuring  an  appropriate  level  of 
training  for  staff  that  conduct  safety  reviews  and 
evaluations;  maintaining  adequate  safety  reporting 
systems;  ensuring  compliance  with  safety  standards 
and  regulations;  ensuring  product  safety  and 
reliability;  and  developing  appropriate  safety  planning 
mechanisms. 


NASA’s  performance  plan  includes  an  objective  that 
the  Space  Shuttle  will  continue  with  safety  and 
performance  upgrades.  Phase  1  upgrades  are 
designed  to  improve  safety  and  performance  that  will 
enable  the  shuttle  to  achieve  the  orbital  inclination  and 
altitude  of  the  International  Space  Station.  A  variety  of 
process  improvements  will  be  implemented  to  enhance 
shuttle  safety  and  reliability  and  reduce  costs. 
Performance  targets  include  (1)  having  a  shuttle 
upgrade  program  that  ensures  the  availability  of  safe 
and  reliable  shuttle  system  through  the  International 
Space  Station  era  and  (2)  achieving  seven  or  fewer 
flight  anomalies  per  mission. 


NASA  included  an  objective  to  ensure  the  health, 
safety,  and  performance  of  humans  in  space. 
Performance  targets  include  (1)  developing  medical 
protocols  and  testing  the  capability  of  crews  health 
care  system  as  integrated  in  the  U.S.  laboratory;  (2) 
completing  the  first  phase  of  the  advanced  life  support 
system  integration  test  bed  facility,  which  will  provide 
the  capability  to  conduct  a  series  of  long  duration, 
human  in  the  loop,  advanced  technology  tests  over  the 
next  6  years;  and  (3)  providing  training  to  the 
appropriate  NASA  supervisors  with  specific  emphasis 
on  actions  to  prevent  injury  and  illness  on  the  job. 


NASA  included  an  objective  to  contribute  to  aviation 
safety  by  reducing  aircraft  accident  rates.  The 
performance  target  is  to  demonstrate  a  conceptual 
aircraft  flight  deck  integrated  with  evolving  ground- 
based  runway  incursion  avoidance  technologies 
installed  at  a  major  airport. 

The  plan  discusses  safety  in  the  context  of _ 
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Inspector  General’s  areas  of  concern 

Applicable  references  in  the  fiscal  year  2000 
annual  performance  plan 

performance  measurement  It  notes  that  it  is  important 
to  avoid  evaluating  actual  output  performance  in 
research  and  development  (R&D)  organizations  by 
counting  the  number  of  planned  events  for  the  year 
with  the  number  that  actually  occurred.  The 
“beancount”  approach,  according  to  the  plan,  is  more 
appropriate  to  a  known  manufacturing  environment.  In 
the  high  performance,  high-risk  R&D  environment  that 
characterizes  NASA’s  programs,  NASA  believes  that  it 
is  inadvisable  to  incentivize  on-time  performance  and 
thereby  deemphasize  safety,  quality,  high 
performance,  and  appropriate  risk-taking. 

Year  2000  comDuter  problems 

NASA’s  OIG  has  reported  that  NASA’s  assessment  of 
Year  2000  Y2K  computer  problems  has  major  gaps. 
NASA  has  identified  the  Y2K  problem  as  a  significant 
area  of  management  concern  in  the  annual  Federal 
Managers’  Financial  Integrity  Act  Report.  OIG  audits 
found  gaps  in  NASA’s  guidance  on  cost  estimation, 
documentation  of  its  Y2K  efforts,  and  its  identification 
of  critical  systems. 

None.  However,  a  Y2K  performance  target  was  added 
to  the  final  FY  1999  performance  plan.  The  plan  states 
that  the  agency  will  complete  remediation  of  mission- 
critical  systems  by  March  1999,  consistent  with 
governmentwide  guidelines  for  Year  2000.  NASA 
officials  stated  that  the  Y2K  concern  would  be  resolved 
by  the  end  of  FY  1999  and  thus  is  not  discussed  in  the 
FY  2000  plan. 

Although  we  did  not  characterize  the  Y2K  problem  as 
a  major  management  challenge,  we  raised  a  concern 
similar  to  the  OIG’s  in  our  review  of  NASA’s  FY  1999 
Performance  Plan.  The  concern  related  to  the  failure 
of  the  information  technology  goal  and  measure  to 
address  the  Y2K  problem  as  well  as  any  significant 
information  security  weaknesses.  (GAO/NSIAD-98- 
181) 

Information  Technoloqv  Security  System 

NASA’s  OIG  reported  that  the  IT  security  system  has 
serious  weaknesses.  The  OIG  recommended  that 
NASA  designate  IT  security  as  a  high-risk  area  in  the 
annual  Federal  Managers’  Financial  Integrity  Act 
report  based  on  the  fragmentation  of  the  system,  the 
lack  of  policies  and  guidance,  network  physical  and 
system  security  weaknesses,  the  lack  of  properly 
trained  personnel,  and  lack  of  threat  analysis. 

None.  However,  the  plan  includes  an  objective  to 
ensure  that  information  technology  provides  an  open 
and  secure  exchange  of  information,  is  consistent  with 
agency  technical  architectures  and  standards, 
demonstrates  a  projected  return  on  investment, 
reduces  risk,  and  directly  contributes  to  mission 
success.  The  FY  2000  plan  includes  as  a  target  the 
improvement  of  information  technology  infrastructure 
service  delivery  to  provide  increased  capability  and 
efficiency  while  maintaining  both  a  customer  rating  of 
“satisfactory”  and  costs  per  resource  unit  at  the  FY 

1998  baseline. 

Waste  and  abuse  as  Financial  Management  System 
is  integrated 

NASA’s  OIG  has  reported  concerns  with  waste  and 
abuse  as  NASA  integrates  its  financial  management 
system.  NASA  has  identified  its  financial 
management  environment,  comprising  of 

The  plan  includes  a  performance  target  to  implement 
new  financial  systems  and  business  procedures 
(including  a  new  financial  classification  structure  and 
full  cost  budgeting  and  accounting  procedures)  through 
the  introduction  and  installation  of  the  1FMS.  Following 
completion  of  testing,  the  system  will  begin  to  be 
installed  at  NASA  headquarters  and  field  centers. 
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National  Aeronautics  and 
Space  Administration 

Office  of  the  Administrator 

Washington,  DC  20546-0001 


APR  2  7  1999 


Mr.  Allen  Li 

Associate  Director  Defense  Acquisitions 
United  States  General  Accounting  Office 
Washington,  DC  20548 


Dear  Mr.  Li: 

As  always,  NASA  values  the  inputs  of  the  General  Accounting  Office  and 
endeavors  to  make  use  of  its  observations  in  improving  the  management  of  the 
Agency.  We  appreciated  the  acknowledgment  of  changes  made  to  address 
concerns  with  our  FY  1999  Performance  Plan  and,  generally,  believe  that  the 
report  is  balanced. 

We  are,  however,  concerned  about  a  few  specific  issues  that  relate  to 
questions  about  the  standards  being  used  to  evaluate  compliance  with  the 
Government  Performance  and  Requirements  Act  (GPRA).  Those  few  issues  are 
identified  in  the  enclosure.  It  is  not  unreasonable  to  think  that  these  standards, 
as  well  as  the  Agency  response  to  those  standards,  will  evolve  as  we  all  gain 
experience  with  the  process.  Unfortunately,  it  would  appear  that  the 
expectations  and  standards  are  being  revised  at  a  pace  that  outstrips  NASA’s 
ability  to  conform  with  those  evolving  expectations. 

We  look  forward  to  continuing  to  work  with  you  to  strengthen  our 
implementation  of  the  GPRA.  NASA  staff  spent  approximately  40  hours 
supporting  this  audit. 

Sincerely, 


^RvDailey 

Associate  Deputy  Administrator 

Enclosure 

cc: 

W/Ms.  Gross 
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Enclosure  III 

Comments  From  the  National  Aeronautics  and  Space  Administration 


Enclosure 


Concern  1 

Evaluation  against  the  Inspector  General’s  identification  of  the  top  10  serious 
management  challenges,  issued  on  December  7,  1998.  The  report,  provided  at 
the  August  5,  1998,  request  of  the  House  Majority  Leader,  was  completed  after 
the  September  1998  submission  of  the  Performance  Targets  to  the  Office  of 
Management  and  Budget.  This  particular  set  of  expectations  was  identified  too 
late  to  enable  inclusion  in  the  performance  plan.  The  process  that  NASA  uses  to 
formulate  and  select  its  performance  targets  begins  early  in  the  calendar  year 
and  concludes  with  the  submission  of  our  budget  to  the  Administration. 

Concern  2 

Continued  criticism  relative  to  failure  to  include  explicit  discussion  of  external 
factors  which  could  impact  attainment  of  the  objectives.  OMB  Circular  A-1 1 , 
220.5,  “Relationship  to  strategic  plans”  states,  “Key  external  factors  are  not  a 
specified  element  in  an  annual  performance  plan.  Agencies  may  choose  to 
include  information  on  external  factors  in  their  annual  plan  when  these  may  bear 
directly  on  goal  achievement  for  the  fiscal  year  covered  in  the  plan.” 

Concern  3 

Characterization  of  the  NASA  FY  2000  Performance  Plan  as  providing  a  “limited 
picture  of  intended  performance  across  the  agency."  The  FY  2000  performance 
plan  provided  204  detailed  performance  targets,  covering  all  of  the  Program  & 
Finance  Accounts  and  most  of  the  Budget  line  items,  as  well  as  addressing  all  of 
the  programmatic  elements  of  the  Strategic  Plan.  It  is  not  clear  how  this 
coverage  can  be  described  as  providing  only  a  limited  picture  of  the  Agency’s 
intended  performance. 
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Enclosure  IV 


GAO  Contacts  and  Staff 
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